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ABSTRACT 

The spectral energy distribution (SED) of a galaxy contains information on the galaxy's physical 
properties, and multi-wavelength observations are needed in order to measure these properties via 
SED fitting. In planning these surveys, optimization of the resources is essential. The Fisher Matrix 
formalism can be used to quickly determine the best possible experimental setup to achieve the 
desired constraints on the SED fitting parameters. However, because it relics on the assumption 
of a Gaussian likelihood function, it is in general less accurate than other slower techniques that 
reconstruct the probability distribution function (PDF) from the direct comparison between models 
and data. We compare the uncertainties on SED fitting parameters predicted by the Fisher Matrix 
to the ones obtained using the more thorough PDF fitting techniques. We use both simulated spectra 
and real data, and consider a large variety of target galaxies differing in redshift, mass, age, star 
formation history, dust content, and wavelength coverage. We find that the uncertainties reported 
by the two methods agree within a factor of two in the vast majority (~ 90%) of cases. If the age 
determination is uncertain, the top-hat prior in age used in PDF fitting to prevent each galaxy from 
being older than the Universe needs to be incorporated in the Fisher Matrix, at least approximately, 
before the two methods can be properly compared. We conclude that the Fisher Matrix is a useful 
tool for astronomical survey design. 



1. INTRODUCTION 



The Fisher Information Matrix (FM, |Fisher, R. A. 



timation from the Cosmic Microwave Background (e.g. 
Tegmark et al.|1997| Balbi et al.|2003 ), or distance mea- 



1935 1 is a statistical instrument of paramount impor 
tance in parameter estimation problems. Its function 
is to predict the precision with which the parameters of 
interest can be measured by a given experiment, start- 
ing from the expected observational uncertainties on the 
data. The utility of the FM comes from the fact that it 
can be used to optimize the experimental setup, accord- 
ing to the desired target results, before actually collecting 

the data. 

The Cramer- Rao inequality ( Cramer, H.|1946 



Rao, C. 



R. [1945 ) states that the uncertainties predicted by the 
FM formalism are, in general, optimistic; they set the 
upper limit for the precision that an experiment can at- 
tain. In particular, the FM method is expected to be 
accurate only if the probability distribution is a Gaus- 
sian function of the parameters. Therefore, in order to 
validate the FM method for a specific problem, it is nec- 
essary to compare the predictions obtained via the FM to 
the results obtained by using other statistical techniques 
that do not suffer from this limitation. In many fields of 
Astronomy, the uncertainties predicted by the FM have 
been shown to closely resemble the ones estimated by 
fitting the probability distribution function (PDF). This 
is the case, for example, of cosmological parameter es- 
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surements from B aryon Acousti c Oscillation s (e.g., Seo 



fc Eisenstein|2003 ); see also e.g., Verde 2010 for a review 
In this paper, we study the applicability of the FM 
formalism to the problem of fitting the Spectral Energy 
Distribution (SED) of galaxies. In this case, the planned 
experiment consists of acquiring multi-wavelength pho- 
tometry of galaxies, the uncertainties on the projected 
data are the photometric errors on the flux measure- 
ments in each band (including any systematics) , and the 
parameters to be measured are the physical properties 
of the target galaxies, such as age, stellar mass and dust 
content. Our objective is to demonstrate, for a variety 
of cases, that the uncertainties forecasted by the FM are 
comparable to the ones obtained by running a PDF fit- 
ting algorithm on the same simulated data, and therefore 
the FM predictions can be used as a guideline in survey 
planning. The advantage of using the Fisher Matrix as 
opposed to full PDF fitting is that the former method 
is much faster, allowing one to very quickly predict the 
expected results for many different experimental setups. 

The paper is organized as follows. In Sec. [2] we re- 
view the currently available tools for SED fitting. Sec. 
[3] describes the relevant Fisher Information Matrix for- 
malism. Our main results for the comparison of uncer- 
tainties obtained via the FM and MCMC SED fitting are 
presented in Sec. [4] we consider several cases of galax- 
ies observed with different signal to noise, star formation 
histories, dust content, and wavelength coverage. In Sec. 
[5] we apply GalFish, the Fisher Matrix algorithm we de- 
veloped, to real data, using a catalog of several hundred 
galaxies at redshifts < z < 5 in the Great Observato- 
ries Origins Deep Survey South (GOODS-S) field, and we 
compare the results to those obtained by using GalMC. 
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We provide a worked example of survey optimization in 
Sec. [6j Sec. [7] contains a summary of our findings and 
our conclusions. 

2. STATISTICAL TOOLS FOR SED FITTING 
SED fitting is the procedure of comparing models 



The Fisher Matrix Fa, which enters the second term 



to the observed galaxy SEDs (for reviews, see Gawiser 
2009 Walcher et al. 2010 and references therein). The 
unknown physical properties of the observed galaxy 
can be inferred from the known physical properties of 
the model, once the model that most closely resembles 
the data (the best-fit model) is found. The best-fit 
model can be located easily by using \ 2 minimization 
techniques, i.e. building a grid of models and com- 
puting the \ 2 f° r each model in the grid. However, 
the computation of the uncertainties on SED fitting 
parameters (also called marginalization) involves solving 
multi-dimensional integrals, and grid-based techniques 
are highly inefficient in performing this task. As a 
result, Markov Chain Monte Carlo techniques have 
become increasingly popular in the past few years. In 
MCMC, the parameter space is explored non-uniformly, 
sampling more frequently where the models resemble 
the data more closely. As a result, the process is more 
CPU-efficient, and computing uncertainties amounts 
to performing sums at the locations sampled by the 
Markov chain. Exampl es of MCMC algorit h ms for 
SED fitting i nclude e. 
et al.| J2006D ; INilsson e! al. 



Pan ter et all (|2003|); |Sajina 

i I anMMwT, i i J *■ ' n L i i = 



pOTOf; jSerra et al | |20TTj f 

Acquaviva et al. (2011a|), w< 



Pirzka 



(|20U7|); IConroy fc Gunn 



In 

] 201 lap, we presented our own MCMC 



cquaviva 

fitting algorithm, GalMC, which is publicly available at 
http://www.physics.rutgers.edu/~vacquaviva/web/GalMC.html 
In this paper, we use GalMC to simulate and tit the 
data, and to compute the marginalized probability 
distributions for the SED fitting parameters to be 
compared with the FM predictions. 

3. FISHER MATRIX FORMALISM 

Let us assume that we are observing galaxies in n pho- 
tometric bands, so that the data is a vector containing n 
fluxes 4>° hs n , each observed with photometric error cr£ bs 
= (Ti bs „. Assuming that our models depend on a set of 

m parameters = (9i,.. m ), where we include in this list 
only the parameters we want to measure, the theoreti- 
cal fluxes in the same photometric bands are denoted as 
4> t i.. n {Si,.. m )- For the moment, we assume flat priors in 
the parameters 0i m , so that the posterior distribution 
coincides with the likelihood; we return to this issue later 
in this section. 
The likelihood associated to each model is 



C = 



(1) 



Let 9q be the location of the best-fit model in the pa- 
rameter space. Around # , which is a maximum of the 
likelihood, we can expand In £ in a Taylor series as: 



in the above equation, is defined as: 

_ i d 2 ln£ 
13 ~~\d9~d9~ 



(3) 



The following relation holds for the marginalized error 
on parameter 9i in terms of the Fisher Matrix: 



(4) 



3.1. Computing the Fisher Matrix 

Equation p| offers the most general way to compute 
the Fisher Matrix. However, in our case this formula 
can be simplified considerably by using the reasonable 
assumption that the data obey a Gaussian distribution 
(in other words, that the likelihood can be written as a 
Gaussian function of the o bservable quan tities) . In this 



case, it can be shown {e.g., Heavens||2009 ) that 



1 



^ = 2 Tr[C 9iCC 9jC 

+c- 1 d^ th (e )d j (4> th (9 )) 

+ C- 1 c)^ th (0o)5 J (0 th (^)) 



(5) 



where C is the data covariance matrix, and di — 8/86i\ 
the trace is computed over the length of the data vector. 
In our case, the photometric errors do not depend on the 
location of the fiducial model in parameter space, and 
the first term in the above equation vanishes. The Fisher 
Matrix formalism is able to describe correlations between 
measurements in different bands as non-diagonal terms 
in the covariance matrix. Here we make the simplifying 
assumptions that they are uncorrelated, and the Fisher 
Matrix becomes simply 



V 



1 



t K bs ) ; 



(6) 



As is well-known, the Fisher Matrix does not require the 
use of any data. 

The above formula also allows us to compare the com- 
putational advantage of using the Fisher Matrix to pre- 
dict the uncertainties on SED fitting parameters, with 
respect to MCMC methods. This comparison is only 
valid for the specific issue of comparing estimated errors 
before taking the data, since the Fisher Matrix is not a 
data fitting technique and, unlike MCMC, does not al- 
low one to find the truly best fitting model and associated 
uncertainties. 

In both cases, almost all of the CPU time is spent 
running a Stellar Population Synthesi s (SPS) code, in our 
case the faster version of GALA XEV (iBruzual & Chariot 
2003| |Charlot fc Bruzual||2011[ ) released with GalMC, to 
compute the theoretical model as a function of the SED 
fitting parameters. The first derivatives of the model 
with respect to each parameter can be approximated by 
the finite differences: 



dittida) = (4>t(9o + Aft)- 



(2) 



■A0O)/(2A0O, (7) 

where A#, is a small increment in the direction of the 
i-th parameter. Therefore, each derivative requires com- 
puting two theoretical models, and the Fisher Matrix re- 
quires a total of 2m iterations, where m is the number of 
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S ED fitting parameters. W e ha ve empirically det e rmined 



Acquaviva et al. (2011a) and Acquaviva et al. (2011b) 



that the necessary number of iterations tor GailVKJ to 
converge is, as a rough rule-of-thumb, 10000m, larger by 
a factor of ~ 5000. 

3.2. Caveats 

The purpose of this paper is to verify whether (or under 
what circumstances) the ">" sign in Eq. Q becomes ef- 
fectively a "~" sign. If the uncertainties from the Fisher 
Matrix are systematically much smaller than the ones 
found by the slower but more accurate parameter esti- 
mation methods which are able to reconstruct the form 
of the PDF, the Fisher Matrix cannot be used as a guide- 
line in planning experiments. Furthemore, because the 
Fisher Matrix always assumes a fiducial model, it should 
only be used in planning experiments and it cannot be a 
substitute for data fitting techniques such as MCMC. 

We expect that differences between the uncertainties 
obtained by FM and PDF fitting can arise from (at least) 
three different issues: 

- A non- Gaussian likelihood. From Eq. ([2]), it is clear 
that if the likelihood is not a Gaussian function of the pa- 
rameters, the truncation in the Taylor series is not exact. 
This is different from the aforementioned assumption of 
a Gaussian distribution of the data. The latter assump- 
tion is well justified at high S/N, while the likelihood 
is strictly a Gaussian function of the parameters only if 
the model fluxes are linear functions of the parameters. 
For a simple star formation history (SFH), this can be 
expected to be true for stellar mass, which is an overall 
normalization of the SED, but not, for example, for age 
or dust content. 

- A displacement between best-fit model and fiducial 
model. The Fisher Matrix is computed at some fiducial 
parameter values, chosen to represent a typical target 
galaxy. However, due to the photometric error, the data 
will be in general scattered away from the fiducial values 
of the fluxes, and the best-fit model might not lie at the 
fiducial location in parameter space, as shown in Fig. [II 
If the shape of the posterior varies significantly with the 
location in parameter space, the size of the uncertain- 
ties predicted by the FM at the fiducial values may not 
reflect those at the location of the best-fit model (and 
as a consequence, those found by PDF fitting). In the 
case of real data, this issue can be further worsened by 
template incompleteness, although this is not a specific 
drawback of the FM formalism but rather a consequence 
of assuming a fiducial model. 

- The impact of priors used in PDF fitting. In the 
expression for the Fisher Matrix, we have implicitly as- 
sumed flat priors in the parameters. While this is not 
in general necessary (and, for example, using a uniform 
prior in ln(age) or ln(mass) is straightforward), it is not 
possible to use top-hat priors (define a range) in the FM 
formalism. This issue is relevant especially for the upper 
bound in age corresponding to the age of the Universe. 
Such prior cannot be taken away from the MCMC run 
used for PDF fitting, not only because it is an impor- 
tant piece of knowledge that we want to incorporate in 
the sampling process, but also because theoretical mod- 
els are only available in a limited age range. However, 
there are ways to simula te pr iors of similar effect in the 
FM, as explained in Sec. 4.2 
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Fig. 1. — Mock data, fiducial and best-fit SED for one high-mass 
and one low-mass galaxy at z = 2 considered in the analysis, show- 
ing the signal-to-noise corresponding to the limiting magnitudes 
listed in Table [T] The mock data points are obtained by adding a 
Gaussian scatter of 1 — a amplitude equal to the photometric er- 
ror to the model fluxes; the best-fit model comes from SED fitting 
using GalMC. For the low-mass galaxy the displacement between 
the best-fit and the fiducial model is more pronounced, since the 
signal to noise ratio is lower and the data have less constraining 
power. The filter transmission curves (multiplied by 0.001) are also 
plotted for reference. 

In the next sections we present the results of our careful 
comparison of the uncertainties predicted by FM and 
MCMC SED fitting, commenting on each of these issues. 

4. FISHER MATRIX VS. MCMC - SIMULATED DATA 

We conduct our analysis on different sets of reference 
galaxies. The assumed wavelength coverage and limit- 
ing magn itudes (summ arized in Table [I]) are inspired by 



Fig. 3 of Grogin et al. (2011), which corresponds t o the 
"Wide" program ot the CAIN DELS ga laxy survey (Gro- 
gin et al.|2011 Koekemoer et al.|2011 ), supplemented by 
photometry in ~th"e Spitzer 1RAC 3.6 and 4 5 [im bands 
from GOODS ( |Dickinson fc Giavalisco|2002[) . Therefore, 
the results found here represent what is achievable, us- 
ing public data only, for a galaxy in GOODS-S. We do 
not include any additional calibration uncertainty We 
consider 1 Gyr old galaxies at different redshifts (z — 
1, 2, and 3) with high (M* ~ 9 x 1O 9 M ) and low 
(M* ~ 1.5 x 10 9 M Q ) stellar mass. We marginalize over 
age, total mass M to t (the integral of the star formation 
history over the age of the galaxy, which is the input of 
SPS codes), and dust content parametrized by the excess 
color E(B-V). Fig. [ljshows the simulated data (including 
a scatter of width equal to the photometric error), the 
fiducial model SED, and the best-fit SED for the high- 
mass and low- mass reference galaxy at z = 2, together 
with the filters' transmission curves. 

For all the galaxies described below, we compute the 
derivatives of the SED (convolved with the 9 filter curves) 
using Eq. Q. The derivative with respect to the total 
mass can also be computed analytically since the SED de- 
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Fig. 2. — High- mass galaxies. Comparison of uncertainties 
on SED parameters from the Fisher Matrix prediction (FM, solid 
lines) and GalMC (dashed lines) for high-mass galaxies (M* ~ 
8 x 10 9 M©). Each row corresponds to a different redshift (z = 1, 
2, and 3). The FM is computed at the fiducial value (blue thicker 
line) and at the best-fit model obtained by GalMC when scatter 
is added to the data (thin green line). The PDF from GalMC 
is shown for simulated data without (dashed line) and with (dot- 
dashed line) scatter. The estimates from FM and GalMC are in 
good agreement in all cases; see text for more details. 

pends linearly on the total mass, and we verified that the 
derivatives obtained with the two methods agree. The 
step size is chosen to be 5% of the reference parameter 
value, with the exception of dust-free galaxies, where we 
used values of E(B-V) = 0.01 and -0.01 (which is un- 
physical, and only used in order to correctly evaluate 
double-sided derivatives). We checked that the step size 
is unimportant by choosing different values and verifying 
that the derivatives do not change. 

The MCMC analysis in GalMC is conducted using a 
flat prior in ln(age) and ln(mass). In the Fisher Matrix, 
we used both a flat prior in age and ln(age), and mass 
and ln(mass), finding no or very little (at the percent 
level) dependence on this choice of prior, which won't be 
further discussed. 

4.1. High-mass galaxies 

The first set of galaxies we consider are massive galax- 
ies (M* ~ 9 x 10 9 M©) at the three reference redshifts z 
= 1, 2 and 3, characterized by constant star formation 
(CSF) history and a moderate amount of dust (E(B-V) 
= 0.3). We begin by simulating data with uncertainties 
corresponding to the magnitude limits in Table [T] but 
not adding scatter to the simulated data points. In this 
case, we are testing the first of the "caveats" listed in the 
previous section; we expect the constraints coming from 
GalMC to be equal to or broader than those predicted by 
the FM, as a result of a non-Gaussian PDF which cannot 
be captured by the FM formalism. 

Our results are shown in Fig. [2] The agreement be- 
tween GalMC and FM is very good for z = 1 and z = 
2 galaxies, the main difference being some non-Gaussian 



tails coming from the MCMC analysis. In the case of the 
galaxy at z = 3, the FM is obviously unable to capture 
the bi-modality of the posterior, and the FM moderately 
underestimates the uncertainties (at the 20% level). This 
behavior is probably a consequence of the lower S /N ratio 
due to the increase in the luminosity distance at higher 
redshift. 

Next, we introduce a Gaussian scatter in the mock 
data, of amplitude equal to the photometric error on each 
data point, and use GalMC to fit the new simulated data. 
This setting corresponds to what we can expect with real 
data. Strictly speaking, the FM is expected to represent 
the PDF at the location of the best-fit model, so find- 
ing a difference in the width of the distribution coming 
from MCMC, which uses the simulated data, and the 
FM, which only uses the theoretical models at the fidu- 
cial value, is not in itself worrisome. However, if the PDF 
at the best-fit model is very different from the PDF at 
the fiducial model, the utility of the FM to predict uncer- 
tainties before performing an experiment is diminished, 
since in that case knowledge of the best-fit model is not 
available. 

The results are also shown in Fig. [2j We find a good 
agreement (within 20%) between the width of the dis- 
tribution obtained by GalMC using the data including 
scatter, and the FM previously computed at the fidu- 
cial model values. To verify that the disagreement (and 
in particular, the smaller width of the dust distribution 
reconstructed by GalMC at z = 2) indeed comes from 
the displacement of the best fit model with respect to 
the fiducial one, we also compute the uncertainties com- 
ing from the FM method at the location of the best-fit 
model. In this case the uncertainties from the FM are 
correctly equal to or smaller than the uncertainties com- 
ing from GalMC. In the subsequent analysis, we only 
compare the FM uncertainties evaluated at the fiducial 
model with the distribution obtained by GalMC by fit- 
ting simulated data including scatter, since this is the 
relevant case for survey planning. 

4.2. Low-mass galaxies 

The next set of galaxies that we consider are similar in 
SFH and dust content to the ones previously discussed, 
but have lower stellar mass of ~ 1.5 X 1O 9 M , and 
consequently, lower signal to noise. We expect SED fit- 
ting to be more challenging in this case, since the dis- 
placement between best-fit and fiducial model will in gen- 
eral be larger, as illustrated in Fig. [I] potentially causing 
a sizable disagreement between the width of the PDF at 
the fiducial model evaluated by the FM and the width of 
the "real" PDF reconstructed by GalMC. 

The results are shown in Fig. [3] At redshift z — 1, the 
FM predictions are similar to the results from GalMC, 
although the uncertainty in the mass is underestimated 
by the FM by about 50%. The anticipated difference 
between fiducial model and best-fit model causes the ob- 
served displacement of the peaks in the two curves; how- 
ever, the widths of the two distributions, which represent 
the size of the uncertainties, are very similar. At higher 
redshift, though, the FM appears to overestimate the at- 
tainable precision on all parameters. In this case, the 
difference is caused by the top-hat prior on age (between 
a minimum value of 10 6 yr and a maximum value of 
14 Gyr) used by GalMC. To properly compare the two 
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Fig. 3. — Low-mass galaxies. Comparison of uncertainties on 
SED parameters from the Fisher Matrix prediction at the fiducial 
model, including a broad Gaussian prior on age (blue solid lines) 
or not (red dashed-dotted lines) and GalMC (black dashed line) 
for low-mass galaxies (A/* ~ 1.5 X lO 9 Af ). Mock data include a 
Gaussian scatter. Each row corresponds to a different redshift (z 
= 1, 2, and 3). The MCMC predictions include a top-hat prior on 
age. At redshift z = 1 (first row), the S/N is still good enough that 
the prior is inessential and the FM predictions match the results of 
GalMC, although the uncertainty in mass is underestimated by the 
FM by about 50% . At redshift z = 2 and z = 3 (bottom two rows), 
the prior used in MCMC needs to be incorporated in the FM, which 
would otherwise overestimate the uncertainties. When the prior is 
taken into account, the uncertainties from the two methods are in 
good agreement for all parameters. 

methods, one would need to use the same prior in the FM 
formalism. However, it is not possible to include a flat 
"range" in the FM, while it is easy to implement a Gaus- 
sian prior in any parameter just by adding it to the corre- 
sponding diagonal terms of the Fisher Matrix. Therefore, 
we add a broad Gaussian prior of width A log 10 (age/yr) 
= 1.6 when computing the FM; we verified that the exact 
width of the prior (between 1.3 and 1.75) is not essential. 
This simple addition is able to reproduce the main effect 
of the top-hat prior in age used by GalMC, and when 
the prior is enforced, the uncertainties predicted by the 
FM or found by GalMC are in good agreement. As an 
empirical prescription for FM calculation, we propose in 
general to run the FM both with and without this prior. 
If no dependence on the prior is found, such as, for ex- 
ample, for the low-mass galaxies at z — 1, or for all the 
high-mass galaxies previously discussed (not shown) , one 
can be confident in the FM results. In general, this will 
be the case when the tails of the distribution predicted 
by the FM are far away from the edges of the top-hat 
priors set in GalMC. On the contrary, if the priors set in 
GalMC intersect the distribution predicted by the FM 
at high likelihood values (near the center, rather than 
near the tails), one should keep in mind that these un- 
certainties might be grossly overestimated, and possibly 
use a Gaussian prior in the FM in order to capture the 
main effect of the top-hat prior used by MCMC. GalMC 
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Fig. 4. — Dust-free galaxies. Comparison of uncertainties on 
SED parameters from the Fisher Matrix prediction at the fiducial 
model (blue solid line) and GalMC (black dashed line) for dust- 
free low-mass galaxies (M* ~ 1.5 X 10 9 A/q). Mock data include 
a Gaussian scatter. Each row corresponds to a different redshift 
(z = 1, 2, and 3). Since dust-free galaxies have higher signal to 
noise in the rest-frame UV-optical than their dusty counterparts, 
the constraints are significantly better than those from Fig. [3] and 
the prior on age is unimportant. The slight underestimation of 
uncertainties from FM (order of 30%) in this case might be due 
to the constraint E(B-V) > used by GalMC, which cannot be 
reproduced by the FM. However, this effect is mild and well within 
the uncertainties observed in other cases. 

also uses a top-hat prior in E(B-V) between and 1. 
While we didn't find this prior to be of importance in 
this case, since the posterior distribution intersects these 
extrema at low values of the likelihood, it might be useful 
in the general case to test that the FM prediction does 
not change significantly if a Gaussian prior in E(B-V) is 
used. A special case is represented by galaxies with very 
high (E(B-V) ~ 1) or very low (E(B-V) ~ 0) dust con- 
tent, which we discuss in detail in the next subsection. 

4.3. Low-mass galaxies, dust-free 

Dust-free galaxies present a possible challenge for the 
Fisher Matrix vs MCMC comparison, because the prior 
that forbids negative values of E(B-V), used in MCMC, 
cannot be incorporated in the FM formalism. The latter 
only allows one to introduce priors centered at the fidu- 
cial model, and any Gaussian prior centered at E(B-V) 
= and wide enough not to introduce a spurious extra 
weight at low-dust values does not have any effect on the 
shape of the Fisher Matrix. The effect of this difference 
would be an underestimation of the width of the distri- 
bution by the FM. In fact, all models with E(B-V) < 
are rejected during MCMC sampling, so that more mod- 
els with non-zero amounts of dust become part of the 
Markov chain during the random walk, making the pos- 
terior distribution (which is proportional to the density 
of sampled points) wider than it would be in absence 
of this constraint. While we observe this effect in our 
comparison in Fig. [4j the difference in the distribution 
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Fig. 5. — Star burst galaxies. Comparison of uncertainties on 
SED parameters from the Fisher Matrix prediction at the fiducial 
model (blue solid line) and GalMC (black dashed line) for starburst 
low-mass galaxies (M» ~ 1.5 X 10 Mq). Mock data include a 
Gaussian scatter. Each row corresponds to a different redshift (z 
= 1, 2, and 3). For these galaxies the parameter "age", identified 
with the time from the starburst, is 200 Myr. The results obtained 
by the two methods are in good agreement. For the z = 3 galaxy we 
also show the effect of a broad Gaussian prior in age (red dashed- 
dotted line) and in E(B-V) (green thin solid line). The former has 
a moderate impact on the prediction for age, while the latter is 
basically negligible. The priors have no effect for z = 1 and z = 2 
galaxies, as expected (see discussion in the text). 

recovered by GalMC and the one predicted by the FM 
is still moderate (around 30%), even for these low-mass 
galaxies. We notice in particular that a large part of the 
visually striking difference in the dust distribution at z 
= 2 is attributable to the median value for dust being 
displaced from the fiducial model by about AE(B-V) = 
0.1, while the width of the PDFs from FM and GalMC 
is indeed similar. Unlike the case of their dusty counter- 
parts, in dust-free galaxies the rest-frame UV and optical 
flux is not suppressed, enabling a better determination 
of age and making the use of a prior in age unnecessary. 

4.4. Low-mass galaxies, SSP 

To verify whether the star formation history has an 
impact on the FM vs PDF fitting comparison, we used 
the two methods to estimate the uncertainties on SED 
fitting parameters for low-mass starburst galaxies, often 
referred to as "simple stellar populations" (SSP). In this 
case, we use the terminology "age" slightly inappropri- 
ately, since we are basically only sensitive to the time 
elapsed since the instantaneous burst of star formation 
happened. In this scenario, the rest-frame UV and opti- 
cal radiation coming from massive stars decreases rapidly 
with time, since these stars have shorter life spans, and 
these energetic photons are not replenished by ongoing 
star formation as in the CSF case. One cannot hope to 
recover the galaxy properties well for a burst that hap- 
pened 1 Gyr before observation (i.e. , at age = 1 Gyr). 
Therefore, we take as reference cases galaxies where the 




Logic (Age/yr) Log 10 (M,„,/M 8 ) E(B-V) 



Fig. 6. — No IRAC coverage. Comparison of uncertainties 
on SED parameters from the Fisher Matrix (FM) prediction at 
the fiducial model and GalMC for low-mass galaxies (M* ~ 1.5 X 
10 9 Mq), when the IRAC 3.6 and 4.5 £im bands are not available. 
Mock data include a Gaussian scatter. Each row corresponds to 
a different redshift (z = 1, 2, and 3). As in previous cases, the 
top-hat prior on Age used in GalMC must be taken into account in 
computing the FM predictions, at least at z = 2 and z = 3 (bottom 
rows); the width of the PDFs obtained from the two methods are 
indeed comparable. As expected, the constraints on SED fitting 
parameters become poorer for increasing redshift, since the lack 
of rest-frame NIR information causes a dramatic increase in the 
uncertainties. 

time since the starburst is 200 Myr. The results of our 
comparison, shown in Fig. [5] confirm that the agreement 
between the width of the distributions predicted by the 
FM and computed by GalMC is good (within 20% after 
the prior on age is properly taken into account for the 
higher-redshift objects, with the exception of the dust 
content for the z = 2 galaxy, which is underestimated 
by a factor two). For z = 3 galaxies, we noticed that 
the prior at E(B-V) = used by GalMC "cuts" the FM 
distribution at a high likelihood value. To possibly re- 
produce this behavior in the FM formalism, similarly to 
what we did with age, we introduced an additional wide 
(AE(B-V) = 0.8) Gaussian prior in the dust distribu- 
tion, centered at the fiducial model (E(B-V) = 0.3). We 
found however that the effect of such prior is negligible 
(bottom-right panel of Fig. [5| . 

4.5. Low-mass galaxies, no IRAC coverage 

As a final test case for our comparison, we considered 
low-mass galaxies for which no IRAC coverage is avail- 
able. It is well known that the rest-frame near-infrared 
spectrum is a good tracer of stellar mass, so we expected 
that determining the galaxy mass would be extremely 
challenging, and we aimed to verify that the FM for- 
malism would be able to capture this phenomenon. Our 
results are shown in Fig. [6j At redshift z = 1, the han- 
dle offered by the near-IR YJH coverage is still able to 
guarantee a measurement of stellar mass within a factor 
of two, and the prior in age has a very moderate effect 
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Fig. 7. — Physical properties of the test sample. The dis- 
tributions of mean ages, stellar masses, and E(B-V) values for the 
set of 485 galaxies in GOODS-S used in Sec. [5] reveal the hetero- 
geneity of the sample. The redshift distribution is also shown in 
the rightmost panel. 

on the shape of the FM. At higher redshift, the complete 
lack of constraints on the SED beyond rest-frame optical 
wavelengths causes a dramatic increase in the uncertain- 
ties in all SED fitting parameters. In these cases, impos- 
ing a pr ior in age in the FM formalism, as discussed in 
Sec. 4.2 becomes important. After the prior has been in- 
corporated, we find once again good agreement between 
the prediction of the FM and GalMC's results, although, 
as expected, the parameters are basically unconstrained 
within their ranges. 

5. FISHER MATRIX VS. MCMC: APPLICATION TO DATA 
FROM GOODS-S 

The comparison between the results of the Fisher Ma- 
trix approximation and SED fitting has so far only been 
applied to a set of simulated data. However, when the 
MCMC algorithm is used on real data, we can expect 
further differences to be introduced by template incom- 
pleteness (the fact that real galaxy spectra arc more com- 
plicated than the models we used to describe them) and 
by the residual effect of priors, as noticed above. To in- 
vestigate this issue, we use a sample of 640 galaxies at 
< z < 5 with spectroscopic redshifts in the GO ODS-S 
field, which are part of the catalog presented in |Dahlen| 



et al. (2010). Each SED contains twelve data points. 
vTriTYIMUS (U band), HST/ACS (F435W, F606W, 
F775W, and F850LP bands), VLT/ISAAC (J, H, and 
Ks bands), and the four Spitzer/IRAC channels (3.6, 4.5, 
5.8, and 8.0^m). A 5% flux error in the U, optical and 
NIR bands, and a 10% error in the IRAC bands is added 
in quadrature to the photometric error to account for 
calibration uncertainty and template incompleteness. 

For each of these galaxies, we use GalMC to perform 
MCMC SED fitting and select the objects for which the 
chains pass all the convergence tests, and the reduced x 2 
value is less than 10. In fact, SED fits with very high 
reduced x 2 value indicate a strong template incomplete- 
ness problem, and the MCMC uncertainties are likely 
to be significantly underestimated in these cases. This 
procedure selects 485 objects, characterized by a broad 
range of physical properties; the distribution of the mean 
expectation values of SED fitting parameters and the red- 
shifts for the sample are illustrated in Fig. [7] We proceed 
to compute the uncertainties on the SED fitting parame- 
ters predicted by our Fisher Matrix algorithm, GalFish, 
for this set of galaxies. In each case, we use as reference 
spectrum (around which the derivatives are computed) 
the one of the model characterized by the mean expec- 
tation values of the parameters for that galaxy. The re- 
sults of our test are shown in Fig. [8j which summarizes 
the differences between the uncertainties found by the 
MCMC and the ones predicted by the FM, expressed by 
the quantity (ctmcmc - cfm)/cfm for each of the param- 
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Fig. 8.— GalMC vs GalFish for galaxies in GOODS-S. The 

figure shows the relative difference in the errors on the SED fitting 
parameters from the two methods, as a function of x 2 (first row), 
signal-to-noise ratio in the SED (second row), and star formation 
rate (third row). We do not observe any significant correlation 
between these properties and the ability of GalFish to match the 
uncertainties of the MCMC analysis. 



eters. 

In the case of age, the average value of the relative 
difference is 10%, meaning that if the uncertainty on 
age predicted by FM is 20%, the uncertainty found by 
MCMC would be 18 — 22%. An useful way to summarize 
the overall behavior of the test is the fraction of objects 
for which the absolute value of the relative difference is 
less than 0.5 or less than 1.0, which are the 68 and 91% 
of the sample. One possible caveat associated to this 
statistic is that objects for which ctmcmc << cfm would 
still satisfy the condition |(ctmcmc — ofm)/cfm| < 1-0, 
despite the disagreement between the uncertainties pre- 
dicted by the two methods. However, such occurrences 
(which amount to about 10% of the objects that pass the 
test) are much more likely to be due to MCMC under- 
estimating the uncertainties because of template incom- 
pleteness, rather than to the failure of the Fisher Matrix 
formalism. In fact, if we alleviate the template incom- 
pleteness problem by considering only the subset of SEDs 
for which \r < 3, these occurrences basically disappear. 
We also note that the median value of the relative dif- 
ference is slightly negative (- 5%), while one would gen- 
erally expect this quantity to be positive, since the FM 
should give an optimistic evaluation of achievable uncer- 
tainties. However, for this test we have assumed only an 
extremely broad Gaussian prior in age, which basically 
has no effect on the parameters, while the MCMC uses 
a top-hat prior on this quantity. Therefore, this result is 
not problematic. 

The results for the uncertainties in the stellar mass are 
similar to those obtained for the age. In this case, the 
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Fig. 9. — Optimization of planned HST observation, I. Left: The forecasted fractional errors for each of the SED fitting parameter 
and their total for the 9794 configurations considered in the worked example described in Sec. VI. Right: The inset shows the small-scales 
fluctuations of the errors in the region where they are minimal; the circle shows the location of the absolute minimum of the sum of the 
fractional errors, whose configuration is described in the text. 



average relative difference (ctmcmc — ^fmI/cfm is 25%, 
the fraction of objects for which the absolute value of the 
relative difference is less than 0.5 or less than 1.0 is 63 and 
85% respectively, and the median value of the relative 
difference is again —5%. The trend is exactly equivalent 
to what is found for the age, as expected since there is 
a well-known positive correlation in the increasing age 
vs increasing mass direction for constant star formation. 
Finally, the dust content parameterized by E(B-V) is the 
parameter for which the FM performs best; the average 
relative difference between the uncertainties found by the 
two methods is 10%, the median disagreement is 1.5%, 
and the fraction of objects for which the absolute value 
of the relative difference is less than 0.5 or less than 1.0 
is 75% and 96% respectively. These statistics are also 
reported in Table [2j 

We also investigate whether the capability of GalFish 
to match the uncertainties obtained via MCMC is cor- 
related with some properties of the target galaxies. In 
particular, in Fig. [8] we show the relative differences as 
a function of the \ of the fit, of the signal-to-noise ra- 

Z-^Z = 1, Tlbands \"i ) 

of the star formation rate of the galaxy, but we do not 
observe any significant trend. Further tests done on the 
correlations of each parameter with the difference in its 
own uncertainty similarly do not reveal any recognizable 
correlation. 



tio of the SED, defined as 



0.5 



and 



6. 



SURVEY PLANNING WITH GALFISH: A WORKED 
EXAMPLE 



In this section we give a practical example of how to 
optimize observations using GalFish. We assume that 
Spitzer-IRAC observations at the depths of GOODS-S 
are available for our target galaxies; we fix the depth 
of the se channels to the ones described in lDahlen et al.l 
(2010) since limited improvements in these measure- 



ments will be available in the near future. Our scope is 
to plan HST observations in the B, V, I, and z channels 
on the Advanced Camera for Surveys (ACS) and the Y, 
J and H channels on the Wide Field Camera 3 (WFC3), 
with a total of 14 requested orbits. Overhead times are 
not included in this total time. We require that at least 
five out of these seven filters must be used, that an in- 
teger number of orbits is spent in each filter, and that a 



maximum of six orbits are spent in one single filter; these 
requirements are merely dictated by common sense and 
serve to limit the total number of possibilities while pre- 
serving the freedom intrinsic in this optimization prob- 
lem. Each optimization process is valid for a certain type 
of target galaxy; in this example, we consider a low mass 
(M„ - 10 9 M Q ), 1 Gyr old, moderately dusty (E(B-V) 
= 0.2) galaxy at z = 2, and we run GalFish on each of 
the 9794 possible observational configurations. In Fig. 
[9j we show the predicted fractional uncertainties on age, 
mass and dust content, as well as their sum, which might 
be used as the overall quantity to minimize. When this 
indicator is used, the favored configuration is one where 
one orbit is allotted to the B, V, and I, and Y filters, four 
orbits are spent in the z filter, six orbits are allotted to 
the J filter, and the H filter is not used. The resulting 
best-case uncertainties achievable on each parameter are 
AAge/Age = 70% or AAge = 0.7 Gyr, AM*/M* = 30% 
or AM* = 3x10 s M , and AE(B-V)/E(B-V) = 30%, 
or AE(B-V) = 0.06. It is interesting to note that the 
configurations that lead to the lowest fractional error in 
each parameter are roughly the same as the ones that 
give the lowest sum; in particular, the aforementioned 
setup is also optimal to minimize the uncertainty on age 
and mass, and very slightly sub-optimal (with a differ- 
ence at the percent level) to minimize the uncertainty on 
E(B-V). 

For all parameters, we observe a "large scale" modula- 
tion which is set by the amount of time spent in the ACS 
B and V filters; better results are always obtained when 
one and only one orbit of the total 14 is dedicated to each 
of these two filters. The only other constant requirement 
observed in the minimal-errors region is that the time 
spent observing with the WFC3 Y filter is between one 
and three orbits. After these variables have been fixed, 
several hundred configurations generate similar predicted 
results; the 50 configurations generati ng the lowest error 



for each parameter are shown in Fig. 10 



7. DISCUSSION 

In this paper we have compared the uncertainties on 
SED fitting obtained by using two different methods, 
the Fisher Matrix formalism and a MCMC algorithm 
for PDF fitting, GalMC. In the first case, the forecasted 
uncertainties come from a mathematical approximation 
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Fig. 10. — Optimization of planned HST observation, II. The three panels show the distribution of time, in orbits, allotted to each 
of the seven niters for the 50 configurations that minimize the error the SED fitting parameters. In all cases, the error is lower when one 
orbit is spent in each of the B (ACS, 435W) and V (ACS, 606W) filters, and a time between one and three orbits is spent in the Y (WFC3, 
105) filter, while various time allocations for all the other bands give rise to similar errors. 



of the posterior distribution function at some fiducial 
value of the parameters; no data are involved. In the sec- 
ond case, the data are used to properly reconstruct the 
PDF by sampling the parameter space through a Markov 
chain. The latter method is much slower but more ac- 
curate; in particular, the width of the PDF predicted by 
the FM is in general predicted to be smaller than the real 
precision attainable on the parameters measured by data 
fitting techniques such as MCMC. However, if it can be 
verified that the uncertainties obtained via the two meth- 
ods are comparable, the much faster FM becomes a very 
useful tool to predict the results of a certain experimental 
setup and to plan astronomical observations. 

We have compared the PDFs predicted by the FM and 
fitted by GalMC for several sets of reference galaxies, 
both on simulated data and real data. The comparison 
that uses simulated data is the most relevant one for the 
purpose of comparing the two methods in the context of 
planning observations, since in any case a fiducial model 
needs to be chosen, whether the FM or MCMC is used to 
compute the uncertainties on the SED fitting parameters. 
On the other hand, performing the comparison on real 
data allows one to test whether the simplifying assump- 
tions used in modeling the stellar populations invalidate 
the attempt itself to predict the achievable precision in 
the measurement of parameters. 

In the case of simulated SEDs, we started by consider- 
ing high-mass galaxies (M* ~ 9 x 1O 9 M ), but most of 
our analyses have been conducted on low-mass galaxies 
(M» ~ 1.5 x 1O 9 M ), for which more potential sources of 
disagreement are present as a consequence of the lower 
signal to noise ratio. We considered dusty and dust-free 
galaxies, with constant and instantaneous star formation 
history with and without infrared photometric coverage, 
at redshifts z = 1, z = 2, and z = 3. We found a remark- 
ably good agreement between the results from the two 
methods. Of course, the FM is not able to capture fea- 
tures such as non-Gaussianity or multi-modality of the 
posterior, which MCMC can correctly explore. However, 
the precision in the SED fitting measurements for the two 
techniques are within 30% of each other (with the FM 
always being an underestimate) in the vast majority of 
cases, and within a factor of two in all cases. Priors play 
an important role in the comparison, in particular for 
higher redshift galaxies with low S/N. MCMC uses a top- 



hat prior in age, (6 < log ln (age/yr) < 10.13), bracketed 
by the minimum age for which templates are trustworthy 
and by the Universe' age. This prior needs to be incor- 
porated in the FM at least approximately, otherwise the 
FM may grossly overestimate the uncertainties. We have 
found that if the fiducial model is far from the extremum 
values of the age range, a good proxy for the top-hat prior 
in age is a Gaussian prior of width Aln(age/yr) ~ 3.5. 
As a general prescription, one can always test the pre- 
dictions with or without the prior in age, but it is easy 
to verify that the prior only has a non-negligible effect 
if the prior intersects the distribution predicted by the 
FM at high likelihood values. We also investigated the 
effect of the range imposed in E(B-V) (between and 1 
in GalMC), finding this prior to be negligible for all cases 
considered. 

We then proceeded to test the performance of the FM 
approximation for a set of 485 galaxies at redshifts be- 
tween and 5 observed in GOODS-S, with widely spread 
photometric and stellar population properties. In this 
case, we also found that the FM provides an acceptable 
approximation to the probability distributions found by 
GalMC, with average relative differences between results 
of the order of 10 — 20%, and significant failures (errors 
differing by more than a factor of two) in ~ 10% of all 
cases. As mentioned earlier, these difference do not nec- 
essarily arise from the failure of the FM formalism, but 
can come from the fact that a real galaxy SED might be 
more complicated than the model used for the analysis, 
and the fiducial model used to predict the uncertainties 
might be significantly different from the observed spec- 
trum. 

In this analysis, we chose to only marginalize over three 
SED fitting parameters: age, mass, and dust content. It 
is often the case that other parameters are considered in 
SED fitting, for example, the e-folding time r for expo- 
nential star formation histories, the metallicity Z, or the 
redshift. As far a s r is concerned, we showed in |Acqua^| 



viva et al. (2011a I that for exponential SFHs the param- 
eters age and r are nearly degenerate, and that different 
choices of prior on r can strongly influence the results of 
SED fitting, even for those data, which have deeper pho- 
tometry than the ones considered in this paper. There- 
fore, r is not a suitable parameter for FM analysis. How- 
ever, an exponentially declining or decreasing SFH with 
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r << age is effectively a starburst either at the age of 
the galaxy or at very recent times, and an exponentially 
increasing or declining SFH with r ~ age is well approxi- 
mated by a constant SFH, so that the results found here 
apply to most exponential SFH models. Similarly, the 
SEDs depend on metallicity in a very non-linear manner 
and thus the effect of Z cannot be effectively captured 
in the FM formalism. The present analysis assumes that 
the metallicity of the target galaxies is known, which is 
a fairly common assumption made in the literature when 
analyzing broad-band photometry data. However, if one 
were to relax this hypothesis, the width of the recovered 
PDFs c ould increase. Our analysis in | Acquaviva et al.] 
( 2011b ) showed that this effect is mild in the case of z ~ 2 
and z ~ 3 Lyman Alpha Emitters with UV, optical, NIR 
and IRAC photometry, with probability distributions be- 
coming ~ 10-20% broader, but of course this effect might 
be stronger for different stellar populations and data sets 
and this caveat should be kept in mind. 

In the case of redshift, a direct comparison with GalMC 
is not really informative, since for real data the redshift 
is usually determined by using photo-z algorithms. We 
have been so far very cautious in using GalMC to de- 
termine photometric redshifts. In fact, when the PDF 
has multiple and widely separated peaks, even MCMC 
codes might fail to explore the parameter space in a way 
that respects the basic MCMC property that the density 
of visited points is proportional to the volume enclosed 
in the probability distribution function. As a result, the 
analysis presented here is strictly valid only if spectro- 
scopic redshifts are available. However, the FM formal- 
ism is able to include redshift as one of the SED fitting 
parameters, at least in the particular form of a Gaussian 
prior centered at the photometric redshift of the galaxy. 
If the uncertainties on the photo-z can be expressed in 
this form, and an educated guess of the width of such 
prior is available, we can expect the uncertainties from 
the FM to be reliable. 

In summary, we have demonstrated that the Fisher 



Matrix formalism is a valuable tool for planning photo- 
metric observations. Estimates of the attainable preci- 
sion on SED fitting parameters are relatively accurate 
(within a factor of two at the maximum) and can be 
obtained very quickly, allowing one to optimize planned 
surveys by simulating a variety of experimental setups. 
Just as one example of the many applications, suppose 
that the scientific goal of an experiment is to achieve a 
certain precision in the measurement of stellar mass. The 
FM formalism allows one to explore what wavelength 
coverage and depth of observations is needed, and to un- 
derstand how the aim can be achieved by minimizing 
costs. 

In the hope that the formalism cast here is useful 
to the scientific community, the Fisher Matrix code 
presented and used here is publicly available as a PHP 
website at http: / /galfish. physics. rutgers.edu| Users can 
specify which filters they want to use from a list of 
many common ones, the depth of planned observations, 
and the characteristics of the target galaxies, and 
they will immediately obtain the results of the FM 
analysis. The Fortran code used in the more complex 
optimization problems will also be available on request 
upon publication of the paper. 
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3 


0.013 


0.024 


0.023 


-0.066 


0.013 


-0.053 


-0.065 


0.066 


0.029 


No IRAC, z = 


1 


0.06 


0.086 


0.17 


0.27 


0.33 


0.42 


0.59 






No IRAC, z = 


2 


0.009 


0.014 


0.018 


0.023 


0.033 


0.059 


0.084 






No IRAC, z = 


3 


0.0014 


0.0045 


0.0069 


0.0082 


0.0095 


0.012 


0.022 






5cr limit [mag] 




26.4 


26.8 


26.2 


25.6 


26.2 


25.6 


25.6 


26.0 


25.4 


lcr flux error \pJy] 


0.02 


0.014 


0.024 


0.042 


0.024 


0.024 


0.042 


0.029 


0.05 



TABLE 1 

Fluxes, 5<j limiting magnitudes, and la errors in /j,Jy used for SED fitting on simulated galaxies with GalMC and for. 
Fisher Matrix forecasts. The simulated fluxes include a photometric Gaussian scatter of amplitude equal to the error. 



Quantity 




Average 


Median 


Standard 
Deviation 


Fraction of objects 
for which \°mcmc-<?fm\ <0 5 

&FM 


Fraction of objects 
for which \°mcmc-°fm\ < L0 

&FM 


°~FM 
( a MCMC —&FM ) 

°~FM 
{^MCMC—VF M ) 

&FM 


, Age 


0.1 


-0.05 


0.63 


68% 


91% 


,M. 


0.26 


-0.05 


0.99 


63% 


85% 


, E(B-V) 


0.1 


0.015 


0.5 


75% 


96% 



TABLE 2 

The relative difference (°'mcmc-< t fm) in the uncertainty on each SED fitting parameter found by the Fisher Matrix and 

by GalMC describes the disagreement between these two methods. Here we summarize the statistical properties of the 
distribution of the differences shown in Fig. [8] which refer, to a set of 485 GALAXIES observed in GOODS-S. 



